Very cool, I'm really glad this seems to have worked out without too much trouble!

This hard-codes the configuration values (max reuse count = 128, max concurrent reuse count = 16, idle timeout = 1s) for now. Once we've decided where these values should be configured (e.g. in the spin.toml manifest, in the runtime
config, or at runtime via the component itself), we can support that. See WebAssembly/WASI#784 for related discussion.

I think this should be configurable via runtime config, plus env vars and/or CLI options, but not spin.toml: the runtime should set its upper limits for reuse / timeouts, with individual components being able to reduce down from there by signaling backpressure / exiting.

The wasi-http discussion is of course relevant, but I think shouldn't really change anything about these config settings in and of themselves.

At one point we discussed exposing app developers to a mix of "new instance" and "reused instance" in the CLI in order to help developers detect latent reuse bugs, e.g. by randomly deciding to biff the current instance, or by having a default reuse configuration that developers would hit a mix in any non-trivial testing. Is that on the agenda?

oh, good point: I had forgotten about that. And yes, we should definitely make these values configurable as ranges, within which we randomize

Sounds good. I'm on a roll right now adding async support to componentize-py, but when I reach a stopping point I'll come back to this and add runtime config support, including ranges for random selection.

I just pushed an update that adds the configuration support we discussed. I noticed that the existing mechanism to configure trigger-http is via clap CLI arguments rather than a runtime config file, so I followed that precedent.

@dicej do you have notes for a reviewer on how to test the different behaviors added here?
And a sample app that would be best to try out?

Thanks!

@dicej do you have notes for a reviewer on how to test the different behaviors added here? And a sample app that would be best to try out?

I did most of my testing using hey and hello-wasip3-http. For example:

1. Install Spin from this PR branch
2. In a clone of the hello-wasip3-http repo, run e.g. spin up --build --max-instance-reuse-count 1..128 --max-instance-concurrent-reuse-count 1..16 --request-timeout 4..8s --idle-instance-timeout 1..8s
   • or spin up --build --max-instance-reuse-count 256 --max-instance-concurrent-reuse-count 7 --request-timeout 8s --idle-instance-timeout 2s, etc.
3. In another terminal run e.g. hey -z 10s http://0.0.0.0:3000/hello

I also tested a variation by adding a wasi::clocks::monotonic_clock::wait_for(40_000_000).await (i.e. sleep for 40ms) to the hello-wasip3-http handle function prior to returning a response. That's necessary to make concurrent reuse kick in, since otherwise each invocation returns immediately without blocking.

When I was testing, I cheated by patching the wasmtime-wasi-http dependency with some extra logging to tell me how many times each instance was reused, what the max concurrency was per instance, etc. Alternatively, I could have added logging to the guest instead, using global variables to track serial and concurrent reuse.

Hope that helps. Happy to provide more detail if needed.